Adversarial training has been empirically shown to be more prone to overfitting than standard training. The exact underlying reasons still need to be fully understood. In this paper, we identify one cause of overfitting related to current practices of generating adversarial samples from misclassified samples. To address this, we propose an alternative approach that leverages the misclassified samples to mitigate the overfitting problem. We show that our approach achieves better generalization while having comparable robustness to state-of-the-art adversarial training methods on a wide range of computer vision, natural language processing, and tabular tasks.

We study the problem of profiling news media on the Web with respect to their factuality of reporting and bias. This is an important but under-studied problem related to disinformation and "fake news" detection, but it addresses the issue at a coarser granularity compared to looking at an individual article or an individual claim. This is useful as it allows to profile entire media outlets in advance. Unlike previous work, which has focused primarily on text (e.g.,~on the text of the articles published by the target website, or on the textual description in their social media profiles or in Wikipedia), here our main focus is on modeling the similarity between media outlets based on the overlap of their audience. This is motivated by homophily considerations, i.e.,~the tendency of people to have connections to people with similar interests, which we extend to media, hypothesizing that similar types of media would be read by similar kinds of users. In particular, we propose GREENER (GRaph nEural nEtwork for News mEdia pRofiling), a model that builds a graph of inter-media connections based on their audience overlap, and then uses graph neural networks to represent each medium. We find that such representations are quite useful for predicting the factuality and the bias of news media outlets, yielding improvements over state-of-the-art results reported on two datasets. When augmented with conventionally used representations obtained from news articles, Twitter, YouTube, Facebook, and Wikipedia, prediction accuracy is found to improve by 2.5-27 macro-F1 points for the two tasks.

Adversarial training is widely acknowledged as the most effective defense against adversarial attacks. However, it is also well established that achieving both robustness and generalization in adversarially trained models involves a trade-off. The goal of this work is to provide an in depth comparison of different approaches for adversarial training in language models. Specifically, we study the effect of pre-training data augmentation as well as training time input perturbations vs. embedding space perturbations on the robustness and generalization of BERT-like language models. Our findings suggest that better robustness can be achieved by pre-training data augmentation or by training with input space perturbation. However, training with embedding space perturbation significantly improves generalization. A linguistic correlation analysis of neurons of the learned models reveal that the improved generalization is due to `more specialized' neurons. To the best of our knowledge, this is the first work to carry out a deep qualitative analysis of different methods of generating adversarial examples in adversarial training of language models.

Graph Neural Networks (GNNs) have been taking role in many areas, thanks to their expressive power on graph-structured data. On the other hand, Mobile Ad-Hoc Networks (MANETs) are gaining attention as network technologies have been taken to the 5G level. However, there is no study that evaluates the efficiency of GNNs on MANETs. In this study, we aim to fill this absence by implementing a MANET dataset in a popular GNN framework, i.e., PyTorch Geometric; and show how GNNs can be utilized to analyze the traffic of MANETs. We operate an edge prediction task on the dataset with GraphSAGE (SAG) model, where SAG model tries to predict whether there is a link between two nodes. We construe several evaluation metrics to measure the performance and efficiency of GNNs on MANETs. SAG model showed 82.1 accuracy on average in the experiments.

Over-approximating the reachable sets of dynamical systems is a fundamental problem in safety verification and robust control synthesis. The representation of these sets is a key factor that affects the computational complexity and the approximation error. In this paper, we develop a new approach for over-approximating the reachable sets of neural network dynamical systems using adaptive template polytopes. We use the singular value decomposition of linear layers along with the shape of the activation functions to adapt the geometry of the polytopes at each time step to the geometry of the true reachable sets. We then propose a branch-and-bound method to compute accurate over-approximations of the reachable sets by the inferred templates. We illustrate the utility of the proposed approach in the reachability analysis of linear systems driven by neural network controllers.

Recently the focus of the computer vision community has shifted from expensive supervised learning towards self-supervised learning of visual representations. While the performance gap between supervised and self-supervised has been narrowing, the time for training self-supervised deep networks remains an order of magnitude larger than its supervised counterparts, which hinders progress, imposes carbon cost, and limits societal benefits to institutions with substantial resources. Motivated by these issues, this paper investigates reducing the training time of recent self-supervised methods by various model-agnostic strategies that have not been used for this problem. In particular, we study three strategies: an extendable cyclic learning rate schedule, a matching progressive augmentation magnitude and image resolutions schedule, and a hard positive mining strategy based on augmentation difficulty. We show that all three methods combined lead up to 2.7 times speed-up in the training time of several self-supervised methods while retaining comparable performance to the standard self-supervised learning setting.

Channel charting (CC) is an unsupervised learning method allowing to locate users relative to each other without reference. From a broader perspective, it can be viewed as a way to discover a low-dimensional latent space charting the channel manifold. In this paper, this latent modeling vision is leveraged together with a recently proposed location-based beamforming (LBB) method to show that channel charting can be used for mapping channels in space or frequency. Combining CC and LBB yields a neural network resembling an autoencoder. The proposed method is empirically assessed on a channel mapping task whose objective is to predict downlink channels from uplink channels.

As the interest to Graph Neural Networks (GNNs) is growing, the importance of benchmarking and performance characterization studies of GNNs is increasing. So far, we have seen many studies that investigate and present the performance and computational efficiency of GNNs. However, the work done so far has been carried out using a few high-level GNN frameworks. Although these frameworks provide ease of use, they contain too many dependencies to other existing libraries. The layers of implementation details and the dependencies complicate the performance analysis of GNN models that are built on top of these frameworks, especially while using architectural simulators. Furthermore, different approaches on GNN computation are generally overlooked in prior characterization studies, and merely one of the common computational models is evaluated. Based on these shortcomings and needs that we observed, we developed a benchmark suite that is framework independent, supporting versatile computational models, easily configurable and can be used with architectural simulators without additional effort. Our benchmark suite, which we call gSuite, makes use of only hardware vendor's libraries and therefore it is independent of any other frameworks. gSuite enables performing detailed performance characterization studies on GNN Inference using both contemporary GPU profilers and architectural GPU simulators. To illustrate the benefits of our new benchmark suite, we perform a detailed characterization study with a set of well-known GNN models with various datasets; running gSuite both on a real GPU card and a timing-detailed GPU simulator. We also implicate the effect of computational models on performance. We use several evaluation metrics to rigorously measure the performance of GNN computation.

大多数流行的大数据分析工具都会发展为改编其工作环境，以从大量非结构化数据中提取有价值的信息。数据挖掘技术从大数据中过滤这些有用信息的能力导致了大数据挖掘一词。将数据范围从小型，结构化和稳定的数据转移到庞大的量，非结构化和快速变化的数据带来了许多数据管理挑战。由于建筑限制，不同的工具以自己的方式应对这些挑战。根据手头任务选择正确的数据管理框架时，有许多参数需要考虑。在本文中，我们为两种广泛使用的大数据分析工具（即Apache Spark和Hadoop MapReduce）提供了一个综合基准，即共同的数据挖掘任务，即分类。我们采用多个评估指标来比较基准框架的性能，例如执行时间，准确性和可扩展性。这些指标专门用于衡量分类任务的性能。据我们所知，文献中先前没有研究所有这些指标，同时考虑到特定于任务的问题。我们证明，在训练模型上，Spark的速度比MapReduce快5倍。然而，当输入工作负载较大时，火花降解的性能。通过其他簇扩展环境可以显着提高火花的性能。但是，在Hadoop中未观察到类似的增强。 MAPREDUCE的机器学习实用程序往往比Spark的精度得分更好，例如3％，即使在小型数据集中也是如此。

从机器学习模型中删除指定的培训数据子集的影响可能需要解决隐私，公平和数据质量等问题。删除子集后剩余数据从头开始对模型进行重新审查是有效但通常是不可行的，因为其计算费用。因此，在过去的几年中，已经看到了几种有效拆除的新方法，形成了“机器学习”领域，但是，到目前为止，出版的文献的许多方面都是不同的，缺乏共识。在本文中，我们总结并比较了七个最先进的机器学习算法，合并对现场中使用的核心概念的定义，调和不同的方法来评估算法，并讨论与在实践中应用机器相关的问题。